Skip to content

Add Rescore functionality #1688

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

peermuellerxw
Copy link
Contributor

Closes #1686

  • [ x] You have read the Spring Data contribution guidelines.
  • [ x] There is a ticket in the bug tracker for the project in our issue tracker.
  • [ x] You use the code formatters provided here and have them applied to your changes. Don’t submit any formatting related changes.
  • [ x] You submit test cases (unit or integration tests) that back your changes.
  • [ x] You added yourself as author in the headers of the classes you touched. Amend the date range in the Apache license header if needed. For new types, add the license header (copy from another file and set the current year only).

@pivotal-issuemaster
Copy link

@peermuellerxw Please sign the Contributor License Agreement!

Click here to manually synchronize the status of this Pull Request.

See the FAQ for frequently asked questions.

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Feb 11, 2021
@pivotal-issuemaster
Copy link

@peermuellerxw Thank you for signing the Contributor License Agreement!

@sothawo sothawo removed the status: waiting-for-triage An issue we've not yet triaged label Feb 12, 2021
@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Feb 12, 2021
Copy link
Collaborator

@sothawo sothawo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR, great work, but I think we should take this further. It is focused on the NativeSearchQuery, but I think we should consider the other Query implementations here as well.

Something like having a

public class RescorerQuery {
  Query query; // <-  our Query interface
  ScoreMode scoreMode;
  int windowSize;
}

not using Elasticsearch classes here, but also having our own enum for the ScoreMode. This would allow to not only use NativeSearchQuery but also for example a StringQuery. In the RequestMapper this would then be mapped to the corresponding Elasticsearch classes – there are existing methods to convert a Query to a QueryBuilder.

What do you think about this change?


private final QueryRescorerBuilder queryRescorerBuilder;

public RescorerQuery(QueryRescorerBuilder queryRescorerBuilder) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is leaking the Elasticsearch class into the API

@sothawo sothawo removed the status: waiting-for-triage An issue we've not yet triaged label Feb 12, 2021
@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Feb 12, 2021
@peermuellerxw
Copy link
Contributor Author

Thanks for looking into my PR and for the feedback. Your suggestions do make sense and I am looking to implement them in the following days :)

pemueller added 2 commits March 10, 2021 23:35
…ionality

# Conflicts:
#	src/main/java/org/springframework/data/elasticsearch/core/RequestFactory.java
#	src/main/java/org/springframework/data/elasticsearch/core/query/AbstractQuery.java
#	src/main/java/org/springframework/data/elasticsearch/core/query/Query.java
@peermuellerxw
Copy link
Contributor Author

I updated the PR and changed the RescorerQuery to a "proper" domain object. Let me know if this is what you had in mind and also if you have any other comments/suggestions. @sothawo

@sothawo sothawo removed the status: waiting-for-triage An issue we've not yet triaged label Mar 13, 2021
@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Mar 13, 2021
Copy link
Collaborator

@sothawo sothawo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine, there are just some minor points with null checks. No need for you to fix these, I will do this in a follow-up commit after merging this PR.

@@ -0,0 +1,94 @@
/*
* Copyright 2020-2021 the original author or authors.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Copyright 2020-2021 the original author or authors.
* Copyright 2021 the original author or authors.

Comment on lines +36 to +38
public RescorerQuery(Query query) {
this.query = query;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public RescorerQuery(Query query) {
this.query = query;
}
public RescorerQuery(Query query) {
Assert.notNull(query, "query must not be null");
this.query = query;
}

we must make sure this is never null.

Comment on lines +63 to +66
public RescorerQuery withScoreMode(ScoreMode scoreMode) {
this.scoreMode = scoreMode;
return this;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public RescorerQuery withScoreMode(ScoreMode scoreMode) {
this.scoreMode = scoreMode;
return this;
}
public RescorerQuery withScoreMode(ScoreMode scoreMode) {
Assert.notNull(scoreMode, "scoreMode must not be null");
this.scoreMode = scoreMode;
return this;
}

@@ -3122,7 +3132,67 @@ void shouldReturnHighlightFieldsInSearchHit() {
assertThat(highlightField.get(1)).contains("<em>message</em>");
}

@Test // DATAES-738
@Test
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@Test
@Test // #1686

@@ -511,6 +522,139 @@ private String requestToString(ToXContent request) throws IOException {
return XContentHelper.toXContent(request, XContentType.JSON, true).utf8ToString();
}

@Test
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@Test
@Test // #1686


private QueryRescorerBuilder getQueryRescorerBuilder(RescorerQuery rescorerQuery) {

QueryRescorerBuilder builder = new QueryRescorerBuilder(Objects.requireNonNull(getQuery(rescorerQuery.getQuery())));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of a NullPointerException we should throw an IllegalArgumentException here

@sothawo sothawo merged commit b289d5f into spring-projects:master Mar 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: waiting-for-triage An issue we've not yet triaged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add rescore functionality
4 participants